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WWC Study Reports are intended to support decision making; neither the What Works Clearinghouse 
(WWC) nor the U.S. Department of Education endorses any interventions. No single Study Report should 
be used as a basis for making policy decisions because (1) few studies are designed and implemented 
flawlessly and (2) all studies are tested on a limited number of participants, using a limited number of 
outcomes, at a limited number of times, so generalizing from one study to any context is very difficult. 

To highlight these issues, the WWC Study Reports describe in detail the specifics of each study, focusing 
primarily on studies that provide the best evidence of effects (randomized controlled trials). Systematic 
reviews of the evidence will be conducted to summarize the results of the individual studies. 

See the WWC Brief Study Report (PDF) for a summary of this study. 
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Summary of Results: Brooks (1 999) compared algebra achievement of students enrolled in the 

intervention group (I CAN Learn® classes) to that of students enrolled in 
the comparison group (traditional classes) at pretest, midyear, and 
posttest. Brooks found that students in the I CAN Learn® classes achieved 
higher scores in algebra than students in traditional classes at both 
midyear and posttest, and the difference was statistically significant. The 
results should be viewed with caution because the unit of analysis 
(student) does not match the unit of assignment (classrooms). 






= Meets Evifdence Stan(dards 






= Meets Evidence Standards with Reservations 



O 



= Does Not Meet Evidence Standards 



The What Works Clearinghouse (www.whatworks.ed.aov) was established in 2002 by the U.S. 
Department of Education’s Institute of Education Sciences to provide educators, policymakers, 
researchers, and the public with a central and trusted source of scientific evidence of what works in 
education. Please email all questions and comments to info(a)whatworks.ed.aov . The What Works 
Clearinghouse is administered by the U.S. Department of Education through a contract to a joint 
venture of the American Institutes for Research and the Campbell Collaboration . 



WWC Detailed Study Report 



1 



Intervention: I CAN Learn® 

Operational Features 

I CAN Learn® Algebra, a software-based math 
curriculum for use in grades 7-10, was 
developed by New Orleans-based JRL 
Enterprises. The developer indicates that I CAN 
Learn meets the National Council of Teachers 
of Mathematics standards. The I CAN Learn® 
software is designed to be interactive and to 
allow students to progress through lessons at 
their own pace. Teachers are expected to play a 
role in determining the content of the lesson and 
other aspects of the class. At the beginning of 
the year, the teacher determines many factors 
about the structure of the course, such as 
homework assignments, lesson organization, 
lesson presentation, manipulatives used, 
assessments, and grade evaluations. 

Each I CAN Learn® lesson follows a five-part 
format consisting of pretest, review, lesson 
presentation, quiz, and cumulative review. 

The pretest covers material from the upcoming 
lesson. If students miss one question on the 
pretest, they continue into the current lesson. 

If students get all the questions right on the 
pretest, they may advance to the next lesson if 
the teacher enables the software’s advancement 
option. The developer describes the lessons as 
being grouped together like the lessons in a 
textbook chapter. After they complete a lesson, 
students take a cumulative review of 
the concepts taught, and teachers are able to 
monitor student progress through continuous 
real-time assessment. 

People, Settings, and Timing 

The I CAN Learn® curriculum is designed for 
ethnically diverse, inner-city students in grades 
7-10. The target population in this study is 
students enrolled in grades 7-10 in 21 schools 
(20 parochial, one public) within five school 
districts or parishes in Louisiana. The five 
parishes were Jefferson, Orleans, St. Tammany, 
Washington, and Vermillion. The study 
included students enrolled in honors, gifted, and 
remedial classes. Students in both I CAN 
Learn® and traditional groups were assessed on 
the outcomes measures at midyear and posttest. 



Cost Information 

Brooks (1999) does not provide cost 
information for implementing I CAN Learn® in 
the five school districts or parishes. 

Intended Duration 

I CAN Learn® is a curriculum that consists of a 
number of lessons. However, Brooks does not 
indicate how many lessons students in the I 
CAN Learn® classrooms completed and how 
many lessons must be completed for the 
curriculum to be implemented as intended. The I 
CAN Learn® curriculum was introduced to the 
classrooms during a two-month period in 1998. 
The quantity of implementation — the duration 
and the dosage — is not clear. 

Scientific Rationale 

No scientific rationale is offered for conducting 
the evaluation. Brooks conducted the evaluation 
for the Superintendent of the Jefferson Parish 
Public Schools to determine the effectiveness of 
I CAN Learn® on students’ algebra achievement 
and time-on-task. This public school system had 
received a technology grant to implement this 
demonstration program. 

Overview of the Study 

Purpose 

This evaluation was designed to compare 
students enrolled in I CAN Learn® 

(intervention) classes with those enrolled in 
traditional (comparison) classes on two 
outcomes: time-on-task behavior and algebra 
achievement. 

Intervention Fidelity 

Brooks does not provide any information about 
the implementation of the program beyond the 
program description. There was no measure of 
teacher implementation. However, Brooks does 
mention that teachers received three days of 
training before the start of the 1998-99 school 
year and that weekly support was available 
through JRL, the I CAN Learn® developers. 

Outcome Measures 

Brooks reports three outcome measures in this 
study: time-on-task, algebra achievement test, 
and the Louisiana Educational Assessment 
Program (LEAP). The WWC only reviews 
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measures that assess students’ academic 
achievement; therefore, the time-on-task 
measure is not discussed in this report. 

The algebra achievement test is a 30-item 
criterion-referenced test based on student 
textbooks. The test was developed by the 
evaluation team and administered three times 
(in October 1998, January 1999, and May 
1999). The algebra achievement test had test- 
retest reliability ranging from .69-.V8 and was 
well aligned to the I CAN Learn® intervention. 
The LEAP is a Louisiana-mandated, criterion- 
referenced test for 8th-graders developed by the 
Louisiana Department of Education and 
administered by local school districts. Brooks 
used only the math section of the LEAP; only 
one of the five components of the math section 
contains algebra questions. Thus, as determined 
by the WWC, this test was underaligned to the I 
CAN Learn® intervention (that is, students were 
taught algebra but tested on content other than 
algebra). Therefore, the only target outcome 
measure reviewed in this study report is the 
algebra achievement test. 

Research Design 

The research design of this study is quasi- 
experimental with statistical equating. Brooks 
selected 102 I CAN Learn® (intervention) 



classes and 67 traditional (comparison) classes 
for participation. It is not clear how classes were 
identified for the study or how they were placed 
in groups, although schools that implemented I 
CAN Learn® seem to comprise a convenience 
sample. Brooks attempted to equate groups by 
balancing honors, gifted, and remedial classes. 
Brooks also statistically equated groups by 
using pretest scores in analyses. It is likely that 
teachers and students knew they were in a study 
and knew they were in the intervention group 
because the methods used in I CAN Learn® 
classes were different from those used in 
traditional classes. 

Participant Flow 

Brooks assigned participants to the intervention 
and comparison groups at the level of the 
classroom. At the beginning of the study. 

Brooks reports a total of 169 classes (102 in the 
intervention group and 67 in the comparison 
group). Brooks does not report the size of the 
sample at the correct level (classroom) in the 
analysis of midyear and posttest scores, but the 
report does not mention anything about attrition 
of classrooms. There was attrition at the student 
level, but Brooks does not mention reasons for 
the attrition. (See Eigure 1, Participant Elow). 
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Figure 1. Participant Fiow (Classrooms) 




Purposeful Exclusions 
from Analysis: 

0 participants excluded 
from analysis 




0 



Lost to Analysis: 
The number of 
participants lost to 
analysis is not reported 
(NR) at the classroom 
level 



Analysis Sample 








NR 1 


1 


NR 1 



Note. Participant flow is reported at the unit of assignment (classroom). Brooks does not report the sample size at the level of 
the classroom except at the start of the study. Attrition at the classroom level in the study is not mentioned. At the student 
level, however, attrition occurs. At the start of the study, 3,304 students received I CAN Learn® instruction and 1 ,632 received 
a traditional curriculum. At pretest, only 3,012 I CAN Learn® students were available, representing a loss of 292 students prior 
to the pretest. In the analysis of the algebra achievement at midyear, there were 1 ,997 I CAN Learn® students and 1 ,190 
traditional students. At posttest, there were 2,230 I CAN Learn® students and 1 ,249 traditional students. Brooks does not 
offer reasons for student attrition. 
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Reference Periods 

Brooks indicates that I CAN Learn® was 
introduced in 21 schools in August/September 
1998. The algebra achievement test was 
administered in October 1998 (pretest), January 
1999 (midyear), and May 1999 (posttest). 

Baseline Data 

Brooks does not describe baseline 
characteristics of I CAN Learn® and traditional 
groups except for the grade level of the students. 
(See Table 1.) Brooks also pretested all 
participants on the algebra achievement test and 
used pretest scores in the statistical analyses. 

Statistical Methods 

A t test was conducted to compare student 
performance in the I CAN Learn® intervention 
groups and the comparison groups on the 
algebra achievement test at pretest, midyear, and 
posttest. All analyses showed that I CAN 
Learn® students performed significantly better 
than students in traditional classes at all 
measurement periods. A repeated measures 
analysis of covariance (ANCOVA) was 
performed, using pretest scores to control for 
initial differences between students; midyear 
and posttest scores were the dependent 
variables. Brooks presents midyear and posttest 



means and standard deviations along with the F 
statistics from the ANCOVA analyses. 

Outcomes and Estimation 

Table 2 presents the effects of the I CAN 
Learn® curriculum on algebra achievement at 
midyear and posttest, as reported by Brooks. 

The repeated measures ANCOVA indicated that 
students in the I CAN Learn® classroom s 
performed significantly better than students in 
the comparison classrooms at midyear and at 
posttest. Brooks does not calculate effect sizes; 
however, means, standard deviations, and 
sample sizes are given for the outcome measure 
at the student level. This information is not 
available for the unit of assignment (classroom). 

Table 2 presents unadjusted midyear and posttest 
means and standard deviations. 

Cautions 

There is a mismatch between the unit of 
assignment (classroom) and the unit of analysis 
(student). This poses two issues. First, the results 
should be viewed with caution because of possible 
dependency issues. Second, the WWC is unable 
to determine if attrition occurred at the classroom 
level. The WWC can determine, however, that 
attrition occurred at the student level, as discussed 
in the note for Figure 1 . 



Table 1. Pretest Characteristics of the Study Sample Reported by 
Brooks (1999) 





Intervention 
in = 3,304) 


Comparison 
{n = 1,632) 


Grade 7 


114 


NR 


Grade 8 


1,943 


NR 


Grades 9 and 10 


1,247 


NR 


Pretest mean score 


3.06 in = 3,012) 


2.52 (n = 1,632) 



Note. Pretest characteristics are reported at the student level (this is the unit of analysis used by Brooks). Brooks assigned 
to the intervention and comparison groups at the level of the classroom; however, the number of classrooms in the 
analysis is not reported. Although 3,304 students received I CAN Learn® instruction, only 3,012 pretest scores were 
analyzed. 
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Table 2. Impact Reported by Brooks (1999): Midyear and Posttest 
Results 





Midyear 


Posttest 






Intervention 


Comparison 


Intervention 


Comparison 


Mean {SD) 


3.48 (3.51) 


3.06 (2.81) 


7.50 (4.66) 


6.90 (3.99) 


Sample size 


1,997 


1,190 


2,230 


1,249 



Note. These are midyear and posttest results for the algebra achievement test, calculated at the level of the student by 
Brooks. Brooks does not calculate the results at the level of the classroom. Also, Brooks does not explain why more 
students’ scores were available at posttest than at midyear. 



Intervention Developer Contact 
Information 

JRL Enterprises, Inc. 

3520 General DeGaulle Drive, Suite 1100 
New Orleans, LA 701 14 
(504) 263-1380 
www.icanlearn.com 

Related Studies 

See reports on other studies of I CAN Learn ®. 

See reports on other studies of Middle School 
Math curricula. 



Report Production 

Date created: October 28, 2004 

Topic area reviewed under: Curriculum-Based 
Interventions for Increasing K-12 Math 
Achievement — Middle School 
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WWC study Ratings®: Brooks (1999) 

Causal Validity: Meets WWC Standards with Reservations, Quasi-Experimentai Design with 

Statisticai Equating 

Students in the intervention group (I CAN Learn® classes) were compared to students in the comparison group (traditional 
classes). It is not clear how classes were identified for the study or how they were placed in groups, although schools that 
implemented I CAN Learn® seem to have comprised a convenience sample. Brooks attempted to equate groups by balancing 
honors, gifted, and remedial classes. Students were pretested on algebra achievement, and Brooks used the scores as 
covariates in the analyses. There did not appear to be attrition at the level of assignment (classroom); however, attrition did 
occur at the level of analysis (student). No extraneous events were identified that appeared to confound the intervention’s 
effect. However, it is likely that teachers and students knew what condition they were in because the curriculum is different 
from that traditionally implemented in the classrooms. 



Other Study 
Characteristics 


Study Rating 


Study-Specific Information 


Intervention Fidelity 


• 


The I CAN Learn® intervention meets the definition of Middle School Math. However, 
Brooks does not provide any information about the implementation of the program 
beyond program description. There was no measure of teacher implementation. 


Outcome Measures 


• • 


Two achievement measures were reported in the study: algebra achievement and the 
Louisiana Educational Assessment Program (LEAP). Only the algebra achievement 
measure is targeted in this report because the LEAP measure was underaligned to the 
content of the intervention. Specifically, only a part of the LEAP consists of algebra 
questions, so students are tested on content they were not taught in the I CAN Learn® 
classroom. The algebra achievement measure is properly aligned to the curriculum and 
has adequate reliability. 


People, Settings, and 
Timing 


• 


Although the sample of students was part of the identified middle school math 
population, it was relatively homogeneous other than variation in grade (students in the 
I CAN Learn® and traditional classes were enrolled in grades 7-10) and student ability 
(students in the I CAN Learn® and traditional classes were classified as honors, gifted, 
and remedial). The targeted outcome measure was implemented at a time appropriate 
to capture the effect of the intervention, but the specific content and skills that the 
achievement measure assesses were not reported. 


Testing within 
Subgroups 


• 


The effect of I CAN Learn® on students’ algebra achievement was tested across the 
entire sample, but the effect was not tested within subgroups of the sample or 
variations across settings. 


Analysis 


• 


The unit of assignment (classrooms) was not the same as the unit of analysis (student). 
Although the statistical properties of the data allowed for valid estimates of the effect 
sizes. Brooks does not report the sample sizes at the appropriate level to allow for these 
calculations. 


Statistical Reporting 


• 


The statistical tests are adequately reported, but they are reported at the level of the 
student. Effect sizes for the outcome of interest could be calculated using a standard 
formula if means, standard deviations, and sample sizes were reported at the level of 
the classroom. 



Note. •• Fully meets criteria; • Meets minimum criteria; X Does not meet criteria. 



^ For more information on the criteria used to rate this study, see the WWC Evidence Standards: Middle School Math . 
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